{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started with the LAT\n",
    "\n",
    "If you're just getting started at working with the Fermi satellite, this page aims to provide a single location summarizing the information you will need.\n",
    "\n",
    "However, understanding the Fermi mission, the LAT instrument, and the full scope of data analysis will require a much more thorough investigation of this than can be provided in a single overview.\n",
    "\n",
    "We urge users to take the time to [read](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/) and to [ask questions](https://fermi.gsfc.nasa.gov/ssc/help/) in order to gain a fuller understanding of the benefits and limitations of the LAT data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic Information\n",
    "\n",
    "Here are some things to know before working with LAT data. Detailed discussions of most of these terms are available in the [Cicerone](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/) with the appropriate page linked as its header."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [LAT Data Products](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_Data/LAT_DP.html)\n",
    "\n",
    " The LAT instrument records discrete _events_: individual readouts that are compared with simulations to generate the **reconstructed direction** and **reconstructed energy** for each event.\n",
    "\n",
    "\n",
    " These events are stored in the LAT _events files_, which serve as the primary data files.\n",
    "\n",
    "\n",
    " Event files come in two flavors: _**photon**_ and _**extended**_ files.\n",
    "      The extended files contain additional information about each event.\n",
    "     \n",
    "\n",
    " The position and orientation information for the spacecraft is provided at 30-second intervals in the _**spacecraft**_ file. Most analyses require both event and spacecraft files. The spacecraft file must cover the entire time range included in the events file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [LAT Performance Page](http://www.slac.stanford.edu/exp/glast/groups/canda/lat_Performance.htm)\n",
    "\n",
    " The LAT team determines the quality of each event (how likely is it to be a well-reconstructed gamma ray).\n",
    "\n",
    "\n",
    " They use these quality parameters to separate the data into _**event classes**_, which are selected with specific performance parameters:\n",
    "   * transient (most background)\n",
    "   * source\n",
    "   * clean\n",
    "   * ultraclean (least background)\n",
    "\n",
    "\n",
    " Be aware that the reduction in background comes with a corresponding drop in statistics. Therefore, most analysis types will strike a balance.\n",
    " \n",
    " A second set of quality parameters, _**event types**_, allow you to refine your data selection based on specific characteristics.\n",
    "\n",
    "   These event types divide the full data set into subsets based on the conversion layer (`FRONT` and `BACK`), the quality of the position reconstruction (`PSF0`, `PSF1`, `PSF2` and `PSF3`), and the quality of the energy reconstruction (`EDISP0`, `EDISP1`, `EDISP2` and `EDISP3`).\n",
    "    \n",
    "   The _PSF_ and _EDISP_ event types are separated into quartiles, with the best quartile being number 3. Each subdivision of data uses a different instrument response function (IRF).\n",
    "    \n",
    "   You absolutely cannot mix results for data that has been filtered using different flavor event types."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [LAT Data Preparation](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_Data_Exploration/Data_preparation.html)\n",
    "\n",
    "When you filter data based on information in the events or spacecraft files, you are making cuts to the data. The link provides a recommendation for cuts that are appropriate for a variety of standard analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [LAT IRFs](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_LAT_IRFs/IRF_overview.html)\n",
    "\n",
    "The reconstruction and/or simulation information gets revised every few years to improve the quality of the data.\n",
    "\n",
    "A change in the event reconstruction or the parameters used to generate the event classes is known as a **pass**, and an improvement in the fidelity of the **instrument response** is a **version**.\n",
    "\n",
    "So, for example, `P7_SOURCE_V6` refers to source class data that has used the `Pass7` reconstruction, and `version 6` of the instrument response function.\n",
    "\n",
    "A change in the version affects _ONLY_ the instrument response, and does not affect the underlying data reconstruction or event class designation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [LAT Background Models](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/BackgroundModels.html)\n",
    "\n",
    "The auxiliary analysis files, such as the _Galactic diffuse_ and _isotropic diffuse_ models, are also dependent on the instrument performance.\n",
    "\n",
    "You should be careful to ensure that you are working with a consistent set of data, models, and IRFs. The file and IRF naming conventions have been designed to help you; click on the link for more details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# High-level Information\n",
    "\n",
    "There is an abundance of information available to you, which includes the [documentation](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/) and [high-level data products](https://fermi.gsfc.nasa.gov/ssc/data/access/). Here are some items that may be useful for you, and are worth reviewing before diving into the data analysis:\n",
    "\n",
    "* Most Recent Source Catalogs:\n",
    "    * [Fourth Point Source Catalog](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/8yr_catalog/) (aka 4FGL)\n",
    "    * [Second Pulsar Catalog](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/2nd_PSR_catalog/)\n",
    "    * [First Catalog of LAT GRBs](http://heasarc.gsfc.nasa.gov/W3Browse/fermi/fermilgrb.html)\n",
    "    * [Third Catalog of High-Energy (>10 GeV) Sources](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/3FHL/) (aka 3FHL)\n",
    "    \n",
    "\n",
    "* Quick-Look Data Products:\n",
    "    * [Bright Monitored Sources](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/msl_lc/) — Daily and Weekly light curves for sources that have flared above 1x10-6 ph cm-2 s-1 during the lifetime of the mission.\n",
    "    * [Aperture Photometry Light Curves](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/4yr_catalog/ap_lcs.php) — For all sources in the 3FGL catalog.\n",
    "    * [List of LAT GRBs](https://fermi.gsfc.nasa.gov/ssc/observations/types/grbs/lat_grbs/)\n",
    "    \n",
    "\n",
    "* Available documentation:\n",
    "    * [Cicerone](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/) — In-depth discussion of the mission, the instruments, the data and the analysis methods.\n",
    "    * [Analysis Overview](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/overview.html) — High-level description of the individual Fermitools and their use in standard analyses.\n",
    "    * [Reference Manual](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/references.html) — Online versions of the fhelp (man page) for each tool.\n",
    "    * [Analysis Threads](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/) — Cookbook-like examples of various analyses designed to walk you through the process step-by-step.\n",
    "    * The talks and tutorials from the Fermi Summer Schools are an excellent resource. See the [github repository](https://github.com/FermiSummerSchool/fermi-summer-school)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Starting Data Analysis\n",
    "\n",
    "The instructions below assume you are planning to analyze LAT data from the most recent data release with the most up-to-date information.\n",
    "\n",
    "If you wish to do a non-standard analysis using older data or analysis products, these links might not be appropriate for you."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [Download and Install](https://fermi.gsfc.nasa.gov/ssc/data/analysis/software/) the Fermitools appropriate for your platform.\n",
    "    * If that doesn't work, check the [FAQ](https://fermi.gsfc.nasa.gov/ssc/help/faq.html).\n",
    "    * Still having problems? Email the [Helpdesk](https://fermi.gsfc.nasa.gov/ssc/help/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Download and install any [user-contributed tools](https://fermi.gsfc.nasa.gov/ssc/data/analysis/user/) or other non-Fermi tools you plan to use:\n",
    "    * All users are likely to want to use the XML model generation python script ([download](https://fermi.gsfc.nasa.gov/ssc/data/analysis/user/make2FGLxml.py)).\n",
    "    * For python users, the LAT analysis scripts are very useful, as are the SED scripts ([download](https://fermi.gsfc.nasa.gov/ssc/data/analysis/user/SED_scripts_v13.1.tgz)).\n",
    "    * For those doing time-consuming tasks, the [multi-processor tools](https://fermi.gsfc.nasa.gov/ssc/data/analysis/user/gtapps_mp-1.1.tgz) may help speed things up.\n",
    "    * XSPEC users may wish to use the [_LATspec_ tool](https://fermi.gsfc.nasa.gov/ssc/data/analysis/user/latspec-1.1.2.tar.gz) to generate XSPEC-compatible spectra.\n",
    "    * The [FitsViewer (fv)](http://heasarc.gsfc.nasa.gov/docs/software/ftools/fv/) tool is useful for looking at the contents of a FITS file.\n",
    "    * The [FTOOLS](http://heasarc.gsfc.nasa.gov/docs/software/ftools/ftools_menu.html) are often used for FITS file manipulation.\n",
    "    * SAO's [ds9](http://ds9.si.edu/site/Home.html) image viewer can be very helpful for looking at FITS images.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* There are several places to download your data. (Data are usually available less than 24 hours after being acquired):\n",
    "    * The [LAT Data Server](https://fermi.gsfc.nasa.gov/cgi-bin/ssc/LAT/LATDataQuery.cgi) is used for traditional position queries.\n",
    "    \n",
    "        * **NOTE**: Due to size limitations, the LAT Data Server only contains the _source_, _clean_, and _ultraclean_ event classes.\n",
    "\n",
    "    * You can also get [All-sky Weekly Photon Files](http://heasarc.gsfc.nasa.gov/FTP/fermi/data/lat/weekly/photon/) (_source_ class or better) from the [FTP](http://heasarc.gsfc.nasa.gov/FTP/fermi/data/lat/) site.\n",
    "    \n",
    "    * The FTP site also has [All-sky Weekly Extended Files](http://heasarc.gsfc.nasa.gov/FTP/fermi/data/lat/weekly/extended/), which are the only data files available that contain _transient class_ events (for GRB analysis).\n",
    "    \n",
    "        * **NOTE**: Weekly files can be easily downloaded from the command line using `wget`. Check the [FAQ](https://fermi.gsfc.nasa.gov/ssc/help/faq.html) for the instructions.\n",
    "\n",
    "\n",
    "* Specialized GRB analyses may use the [LAT Low Energy](http://heasarc.gsfc.nasa.gov/W3Browse/fermi/fermille.html) event files rather than the standard event files.\n",
    "\n",
    "\n",
    "* Be sure to review the [*Data Caveats*](https://fermi.gsfc.nasa.gov/ssc/data/analysis/LAT_caveats.html) that the LAT team has provided accompanying each data set.\n",
    "    * You should review these prior to using LAT results in scientific publications. **`IMPORTANT!`**\n",
    "\n",
    "\n",
    "* You should also review the [*Temporal Caveats*](https://fermi.gsfc.nasa.gov/ssc/data/analysis/LAT_caveats_temporal.html) to be sure an apparent signal in your data is not due to mission-related periodic effects.\n",
    "\n",
    "\n",
    "* Don't forget to grab a _spacecraft_ file!\n",
    "    * Also available from the [LAT Data Server](https://fermi.gsfc.nasa.gov/cgi-bin/ssc/LAT/LATDataQuery.cgi).\n",
    "    * You can get [Weekly Spacecraft Files](http://heasarc.gsfc.nasa.gov/FTP/fermi/data/lat/weekly/spacecraft/) from the [FTP](http://heasarc.gsfc.nasa.gov/FTP/fermi/data/lat/) site.\n",
    "    * However, for all-sky analysis, we recommend using the [Mission-long Spacecraft File](http://heasarc.gsfc.nasa.gov/FTP/fermi/data/lat/mission/spacecraft/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Collect any auxiliary analysis files you may need. Many of these files are also available in the `refdata` directory in your Fermitools Installation (`$FERMI_DIR/refdata/fermi/galdiffuse/`).\n",
    "    * [Background model files](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/BackgroundModels.html): You will need a _Galactic diffuse_ model ([description](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/Model_details/FSSC_model_diffus_reprocessed_v12.pdf)) as well as the _isotropic diffuse_ model appropriate to your data set.\n",
    "    * You will likely need to get the most recent Extended Sources Templates ([download zip file](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/4yr_catalog/LAT_extended_sources_v15.tgz)). These templates are usually needed when analyzing a long duration dataset near the Galactic plane, or near the three extragalactic extended sources (LMC, SMC, and CenA).\n",
    "    * You may want to include a custom Solar System Template ([description](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/SSTmodels_description.pdf)) in your analysis to account for emission from the Sun ([download model](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/latSolarTemplate/solar_profile_v2r0.fits)) or the Moon ([download model](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/latSolarTemplate/lunar_profile_v2r0.fits)). There are [detailed instructions](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/solar_template.html) on how to construct a custom template file.\n",
    "    * You may want to use the Earth Limb template (available from the [4-year Catalog page](https://fermi.gsfc.nasa.gov/ssc/data/access/lat/4yr_catalog/)). This model is strictly for point source analysis using the 4-year dataset, and so is unlikely to be useful for most analyses."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Review the analysis instructions appropriate for the science you are planning, as well as the data usage notes specific to your data set ([Pass 8](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Pass8_usage.html), [Pass 7 Reprocessed](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Pass7REP_usage.html), [Pass 7](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Pass7_usage.html)). The most common analysis types are:\n",
    "    * [Binned Likelihood](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/binned_likelihood_tutorial.html) — Most common method, and recommended for most analyses.\n",
    "    * [Unbinned Likelihood](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/likelihood_tutorial.html) — Use for smaller data sets, e.g. monthly light curves, or high-energy events.\n",
    "    * [Python for Likelihood](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/python_tutorial.html) analysis — Python adds flexibility for refits, model manipulation, upper limit calculation, energy dispersion correction, and additional parameter output.\n",
    "    * GRB analysis\n",
    "        * [GBM bursts](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/gbm_grb_analysis.html)\n",
    "        * [LAT bursts](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/lat_grb_analysis.html)\n",
    "        * [Joint GBM and LAT spectral analysis](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/combined_LAT_GBM.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Be sure to review the current [Recommended Cuts](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_Data_Exploration/Data_preparation.html) from the LAT team for your type of analysis. **`IMPORTANT!`**\n",
    "\n",
    "\n",
    "* Additional [Analysis Threads](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/) are available with cookbook-like examples for a wide variety of analyses."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Coordinating Observations\n",
    "\n",
    "The LAT normally operates in all-sky scanning mode rocking north and south on alternating orbits. This nominal [sky-survey mode](https://fermi.gsfc.nasa.gov/ssc/observations/types/allsky/) acquires ∼30 minutes of exposure every 3 hours for every point on the sky with reasonably constant exposure. However, there are [other types of observations](https://fermi.gsfc.nasa.gov/ssc/observations/types/) that are used to enhance exposure on a variety of targets.\n",
    "\n",
    "Timelines are planned three weeks into the future and are planned on _Mission Week_ boundaries (Thursday to Wednesday, in UTC). Timelines are posted publicly as soon as they have been scheduled. Any desired changes to the future observing schedule should be addressed through the GI process (for long term or large-scale observing changes), or as a TOO (for rapid-response within the current observing cycle).\n",
    "\n",
    "Here are some resources to find out what the LAT has been observing, and where it will be pointing in the future, to help coordinate observations at other wavebands.\n",
    "\n",
    "* [Observatory Timelines](https://fermi.gsfc.nasa.gov/ssc/observations/timeline/posting/): Current and future planned timelines (by mission week)\n",
    "    * **NOTE**: You can convert date to/from mission week by using the [xTime](http://heasarc.gsfc.nasa.gov/cgi-bin/Tools/xTime/xTime.pl) utility.\n",
    "    \n",
    "    \n",
    "* Targets of Opportunity:\n",
    "    * [TOO request webform](http://heasarc.gsfc.nasa.gov/ark/fermitoo/help.html)\n",
    "    * List of [approved TOOs](https://fermi.gsfc.nasa.gov/ssc/observations/timeline/too/)\n",
    "    * Timeline of scheduled [TOO observations](https://fermi.gsfc.nasa.gov/ssc/observations/timeline/posting/too/)\n",
    "    \n",
    "    \n",
    "* Request for [coordinated multi-wavelength observations](https://fermi.gsfc.nasa.gov/ssc/observations/multi/reporting/)\n",
    "    * **NOTE**: If you provide your schedule for multi-wavelength observations, the Fermi project will consider the impact on your planned program before approving TOO observations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Guest Investigator Program\n",
    "\n",
    "Proposals for the Fermi Guest Investigator program are usually due mid-January each year. The program funds many different areas of science — data analysis, theory development, multi-wavelength studies, etc. — as long as they can be shown to reasonably relate to the Fermi mission's science goals.\n",
    "\n",
    "* [General information about the GI program](https://fermi.gsfc.nasa.gov/ssc/proposals/)\n",
    "\n",
    "* Most recent [Fermi amendment in NSPIRES](https://nspires.nasaprs.com/external/viewrepositorydocument/cmdocumentid=397753/solicitationId=%7B0A167832-6EEF-26FE-9585-E2DF3EB1AE7A%7D/viewSolicitationDocument=1/D.6%20Fermi%20Amend%2039.pdf)\n",
    "\n",
    "* Joint science opportunities are available through the GI program for time on the following observatories:\n",
    "    * [NOAO](https://fermi.gsfc.nasa.gov/ssc/proposals/noao.html)\n",
    "    * [NRAO](https://fermi.gsfc.nasa.gov/ssc/proposals/nrao.html)\n",
    "    * [Arecibo](https://fermi.gsfc.nasa.gov/ssc/proposals/arecibo.html)\n",
    "    * [Suzaku](https://fermi.gsfc.nasa.gov/ssc/proposals/suzaku.html)\n",
    "    * [VERITAS](https://fermi.gsfc.nasa.gov/ssc/proposals/veritas.html)\n",
    "    * [Integral](https://fermi.gsfc.nasa.gov/ssc/proposals/integral.html)\n",
    "\n",
    "\n",
    "* Previously approved programs:\n",
    "    * [Cycle 7](https://fermi.gsfc.nasa.gov/ssc/proposals/cycle7/ApprovedPrograms.pdf)\n",
    "    * [Cycle 6](https://fermi.gsfc.nasa.gov/ssc/proposals/cycle6/ApprovedPrograms.pdf)\n",
    "    * [Cycle 5](https://fermi.gsfc.nasa.gov/ssc/proposals/cycle5/ApprovedPrograms.pdf)\n",
    "    * [Cycle 4](https://fermi.gsfc.nasa.gov/ssc/proposals/cycle4/AcceptedPrograms.pdf)\n",
    "    * [Cycle 3](https://fermi.gsfc.nasa.gov/ssc/proposals/cycle3/AcceptedPrograms.pdf)\n",
    "    * [Cycle 2](https://fermi.gsfc.nasa.gov/ssc/proposals/cycle2/approved_programs.html)\n",
    "    * [Cycle 1](https://fermi.gsfc.nasa.gov/ssc/proposals/cycle1/Cycle1Approved.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Staying Connected\n",
    "\n",
    "The FSSC hosts a number of [newsletters](https://fermi.gsfc.nasa.gov/ssc/library/newsletter/) that can help you stay in touch with Fermi-related news. Most are very low-frequency, contacting you only when something important needs to be shared (meeting announcements, new software release, etc.). Here are the links to sign up:\n",
    "\n",
    "* [fermi-news](https://lists.nasa.gov/mailman/listinfo/fermi-news) — General news about the Fermi mission, meeting announcements, etc.\n",
    "* [fermi-soft](https://lists.nasa.gov/mailman/listinfo/fermi-soft) — Updates on the analysis software and science data.\n",
    "* [fermi-papers](https://lists.nasa.gov/mailman/listinfo/fermi-papers) — Updates on Fermi related papers. (This list sends frequent emails!)\n",
    "* [gamma-mw](https://lists.nasa.gov/mailman/listinfo/gammamw) — Information on multi-wavelength observations relevant to Fermi.\n",
    "\n",
    "As always, if you have any questions about Fermi data or analysis, first check the [FAQ](https://fermi.gsfc.nasa.gov/ssc/help/faq.html). If your question is not addressed there, please contact the [Helpdesk](https://fermi.gsfc.nasa.gov/ssc/help/), and we'll get back to you as quickly as possible."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.16"
  },
  "nav_menu": {},
  "toc": {
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 6,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
